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Abstract 

The  problem  of  designing  an  optimal  state  estimator  for  a 
linear,  discrete-time  system  with  a  singular  noise  covariance 
matrix  is  considered.  In  this  article,  this  problem  is  cast  as 
a  constrained  optimization  problem  and  the  approach  appears  to 
be  more  direct.  Solution  to  this  optimization  problem  gives  a 
reduced-order  optimal  state  estimator. 


i 


j 


I.  INTRODUCTION; 


In  a  linear  stochastic  system,  the 
output  measurement  may  be  only  partially 
noise  corrupted.  Although,  in  practice, 
one  may  argue  that  there  exist  no  noise- 
free  measurements,  it  is  quite  possible 
that  some  of  the  measurements  are  noise 
corrupted  while  the  others  are  relatively 
accurate.  Under  the  Gaussian  assumption, 
this  implies  that  the  noise  covariance 
matrix  has  both  large  and  small  eigen¬ 
values,  which  easily  leads  to  numerical 
difficulties  in  the  implementation  of  the 
Kalman  filter.  It  is  convenient  in  this 
case  to  model  these  more  accurate  measure¬ 
ments  as  noise- free  entities. 

The  study  of  this  problem  dates 
back  to  the  work  of  Bryson  [1],  for  con¬ 
tinuous-time  systems,  and  that  of  Braramer 
[2],  for  discrete-time  systems. 

KWakernaak  [3]  and  Anderson  [4]  discussed 
this  problem  as  a  singular  linear  state 
estimation  problem;  however,  no  explicit 
solutions  were  given.  Tse  and  Athans  [5) 
derived  a  rather  complicated  "observer- 
estimator"  which  is  essentially  an  exten¬ 
sion  of  the  Luenberger  observer  [6] . 

Later,  Yoshikawa  [7]  gave  a  simpler  deri¬ 
vation  for  minimum- order  optimal  state 
estimators.  More  recently,  Pairman  [8] 
proposed  a  "hybrid  estimator"  which 
features  " coord inatizat ion"  and  achieved 
a  reduced-order  optimal  estimator. 

The  main  feature  of  the  approach 
used  in  this  paper  is  the  following: 

After  a  proper  similarity  transformation, 
the  state  variables  are  decomposed  into  - 


two  parts,  one  to  be  estimated  by  a  reduced- 
order  filter  and  the  other  to  be  recovered 
exactly  from  the  noise- free  measurements. 
Then  the  dynamic  equation  of  the  latter 
part  of  the  state  equation  is  considered  a 
constraint  on  the  optimal  estimation  of  the 
other  part  of  the  states.  Hence  the  state 
estimation  problem  in  this  case  is  cast  as 
a  constrained  optimization  problem,  which 
leads  to  a  reduced-order  optimal  state 
estimator. 


~2.  PROBLEM  FORMULATION 


A  linear,  discrete- time  stochastic 
system  can  be  described  by  the  following 
equations 

x{k+l)«A(k)  x(k)+B(k)  u(k) ,  k-0,1,2,...  (1) 
y(k)  -  C(k)  x (k)  +  v (k)  ,  k-1,2 -  (2) 


where  x(*)«  Rn#  u(*)c  Rp,  and  y(*)«  R1”. 

To  further  specify  the  problem,  the  follow¬ 
ing  assumptions  are  made: 

(i)  x(0) ,  u(0)  ,  u(l),  ...,v(l) ,  v(2), 

. . . ,  are  independent  random  vec¬ 
tors  with  the  following  statistics 


E[x(0H-x0 
B[u(k)  Tk 

*[v(k)]-0  Yk 


E[x(0)xT(0)]«V 

I 

B [u(k) uT(k-i) ] 
-Vu(k)6(i)  Tk,  i 

B[v(k)vT(k-i)J 
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-Vy(k)6(i) 


Tic,  i 


E[u(k)vT(i)]«OpXm  *k,  i  E[u(k)xT(0)] 


E[v(k)x^(0) 


0  Tk 
pxn 


0  v  tk 
m  xn 


where  u  (•)  and  v  (•)  denotes  the  trans¬ 
pose  of  vectors  u(-)  and  v(-)#  respective¬ 
ly,  and  &(•)  deontes  the  Kronecker  delta, 
(ii)  For  any  k,  the  Vv(k)  is  a  non¬ 
negative  definite  matrix  with 
rank  m^,  where  m^<tn.  Under 

this  assumption,  implementation 
of  the  standard  Kalman  filter 
involves  inversion  of  a  matrix 
which  may  be  singular.  Tse 
and  Athans  15]  proposed  an  ob¬ 
server-estimator  of  order  n-m2 

which  performs  as  well  as  higher 
order  estimators,  where  m^^m-m^. 

(iii)  For  any  k,the  C(k)  is  of  full 

rank,  i.e.  every  element  of  the 
output  measurement  is  indepen¬ 
dent  of  the  others. 

The  objective  here  is  to  design  an 
state  estimator  of  order  n-m^. 

can  assume 


vT(. 


optima  1 

Without 

that 


loss  of  generality,  one 


v(k) 


r 

rai  *2 


where  v^(k)  €  R  ,  and  thus  the  covariance 
matrix  of  v(k)  can  be  written  as 

(k)  ■  0. 


Vv(k) 


"v  00  lo 

V1  I  mlxm2 

0  1  0  ~ 

i  in 2  xm2 


1 

where  V  (k)  is  strictly  positive  definite. 


It  is  easy  to  see  that  there  exists 
a  non-singular  matrix  Q(k) ,  such  that  the 


transformation 

z(k) 


Q(k)  x(k) 


(3) 


yields  the  following  state  and  measurement 
equations 


6 


iOt+l)' 

jTkTUj- 


•nd 

[y2TvTj 


Au(k) 

!*u<k»  1 

Ajjtk) 

\\2U)  J 

®l(k,l 

-  -  - 

u(k) 

B2(k)J 

;u<k>_ 

I^12*k)  If 

m2  xn  ^  1 

ic^urji 

r«i<«  1 

[l2TkTj 


(4) 


l<k>| 

,T*T 


A(k)  -  Q(k+1)  A(k)  Q_1(k) 
S(k)  «  Q(k+1)  B(k) 


and 

?(k)  -  C(k)  Q-1(k) 
n« 

z^Oe  R  A.z2(-)e  R  .y^  •)  e  R  1.y2(-)e  R  % 
and  n^n-mj.  X(*K  &(•)  and  ?(*)  are  par¬ 
titioned  accordingly.  Moreover,  since  C(k) 
is  of  full  rank,  ?22(k)  i®  Avertible. 

Hence  there  exists  a  one-to-one  correspon¬ 
dence  between  the  state  z2(k)  and  outPut 

y2(k),  namely 

z2(k)  *  C22(k)  y2(k)  (6) 

and  thus  in  the  state  equation,  (4),  only 
z^(‘)  must  be  estimated.  The  dynamic 

equations  for  2^(0  and  z2(*)  are 

21(k+l)  =  A^kjz^k)  +  A12(k)22(k)+B1(k)u(k) 

(7) 

22(Kfl)  =  A21(k)z1(k)-fA22(k)z2(k)+S2(k)u(k) 

(8) 

It  is  obvious  from  (7)  and  (8)  that  Zj,(k) 
and  22(k)  are  mutually  dependent;  there¬ 
fore  the  estimation  of  z^fk)  does  depend 
on  the  dynamic  behavior  of  z2(k) .  Thus 
the  filtering  problem  becomes  that  of  find¬ 
ing  an  optimal  ^(k+ljk+l)  subject  to  (7) 

and  constrained  by  (8),  where  ^(k+llk+l) 
denotes  the  estimate  of  z^(k+l)  given 

measurements  up  to  time  k+1.  Note,  from 
(6)  and  (7),  that  the  state  z2(k)  can  be 

regarded  as  a  deterministic  input  in  the 
Kalman  filtering  problem. 

S.  THE  REDUCED- ORDER  OPTIMAL 
STATE  ESTIMATOR 

In  this  section,  the  optimal  estima¬ 
tor  for  z^(k)  is  derived  where  the  perform 

mance  measure  is  the  trace  of  the  error 
covariance  matrix.  Defining  the  vector 

•(k)  as 

a  ni«»2 

•  (k)  A  *  (it)  -  P(k)  z,(k)  r  P(k)c  R  1 

1  2  (9) 

from  (S),  (7)  and  (8),  one  obtains 
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a(k+l)-F(k)s(k)+G(k)z2(k)+M(k)u(k)  (10) 

and 

y1(k)*H(k)a(k)+N(k)z2(k)+v1(k)  (11) 

where 


F(k)  - 

*11 (k)  " 

P(k+1) 

A2  2  ( k) 

(10. a) 

Ai2  ( k) 

-  P(k+1) 

A2  2  (k) 

+  F00 

POO 

(10. b) 

M(k) 

-  S^k)  - 

P(k+1) 

B2(k) 

(10. c) 

H(K) 

-  cu(k) 

(11. a) 

N(k) 

-  c12(k) 

+  Cxl(k) 

P(k) 

(11. b) 

Notice  that  P(k)  ,  as  defined  in  the  above 
equations,  can  be  viewed  as  the  Lagrange 
multiplier  in  the  standard  constrained 
optimization  problem.  Now,  the  problem 


of  estimating  z^(k)  is  replaced  by  that  of 
estimating  s(k)  given  the  measurements 


(y1(l),y1(2) . yx(k) }  and  states 

lz2(l). 

z2(2) , . . . ,z2(k) ).  From  (9),  it  is 

obvious 

that 

l(k|k)  -  $,(k|k)  -  P(k)  z2(k) 

(12) 

and 

v~(k|k)  -  v~  (k|k) 
s  zx 

(13) 

where  A(klk)  denotes  the  estimate  of  s(k) 
conditioned  on  input-output  measurements 
up  to  time  k,  and 


s(k|k)  ^  ft(k|k)  -  s(k) 

Zjfklk)  ^  $j(k|k)  -  z1(k) 

V~(k|k)  ^  El^klkji^tklk)] 

The  unbiased  linear  estimator  of  a(k) 
is  given  by  the  following  n^th-order 

filter 

4(k+l) | k+l)« (I-K(k+l)H(k+l) ]P(k) l(k|k) 

♦K(k+1)  [y2(k+l)-N(k+l)  *2(k+l) 
-H(k+l)G(k)  *j(k)  J  (14) 

Benc«  the  error  quantity  s(k|k)  propagates 

aa 

s(k+l|k+l)“  (l-K(k+l)H(k+l)  ]F(k)a(k|k) 

+  (l-K(k+l)H(k+l)  )M(k)  u(k) 
-K(k+l)v1(k+l)  (15) 

and  the  error  covariance  statrix  V~(  •  |  • )  la 
given  by 

V~(k+l|k+l)-lI-K(k+l)H(k+l)  )r(k4l) 

. . II-K(k+l)H(k+l)]T 


+K(k+l)Vv  (k+1)  KT(k+l)  (16) 

where 

r(k+l)^F(k)V~(k|k)FT(k)+M(k)V  (k)MT(k) 

“  U  (17) 

Observe  that  T(k+1)  is  essentially  the 
one-step  prediction  error  covariance  matrix 

[9] .  An  optimal  estimator  is  taken  to  be 
an  estimator  which  minimizes  the  trace  of 
the  error  covariance  matrix.  Therefore, 
it  is  left  to  minimize  Tr  [V~(k+l| k+1)  ] 

with  respect  to  (K(k+1):  P(k+1) ] .  Notice 
that  here  K(k+1)  plays  the  role  of  stand¬ 
ard  Kalman  gain  while  P(k+1)  is  the 
Langrangian  of  the  optimization  problem. 
Minimizing  Tr  [V~(k+l] k+1) ]  with  respect  to 

K(k+1)  yields 

K* (k+1)  -  T(k+1)  HT(k+l)  R_1(k+1)  (18) 

where  R(-)  is  the  positive- definite  symme¬ 
tric  matrix  given  by 

R(k+1)  -  H(k+l)r(k+l)HT(k+l)+V  (k+1) 

V1 

(18. a) 

Observe  that  (15) -(10)  are  identical  to 
the  formulation  of  the  standard  Kalman 
filter  [9).  However,  in  this  case,  it  is 
further  required  to  optimize  the  state 
estimator  with  respect  to  the  choice  of 
P(k+1) ;  i.e.  minimize  Tr  (V~(k+1  |k+l)  ]  with 

respect  to  P(k+1) .  Let  the  optimal  P(k+1) 
which  minimizes  Tr  [V~(k+1 1  k+1)  ]  be  denoted 

by  P*(k+1).  Then  it  can  be  shown  that 

P*  (k+1)  c  I*(k+1)  (19) 

where  I*  (k+1)  is  the  set  given  by 

n. 

Z*(k+1)  -  {P:  P  A,(k)«A,(k).  P  c  R  1 

1  2  (19. a) 

Ax(k)  -  A21(k)V~(k|k)^1(k)+S2(k)Vu(k)B^(k) 

(19. b) 

A2(k)  -  A11(k)Vy(k|k)Xj1(k)+ri(k)Vu(k)B^(k) 

(19. c) 

Notice  that,  when  A^(k)  is  non-singular, 
P*(k+1)  ia  given  by 

P*(k+1)  -  A2(k)  AjX(k)  (19. d) 

The  set  X* (k+1)  will  be  discussed  in  the 
next  section. 

Once  P*(k+1)  is  found,  the  matrices 
F(k) ,  0(k) ,  and  M(k)  can  be  specified  and 
denoted  by  F*(k),  0*(k),  M+(k) *  respective¬ 
ly,  by  substituting  P*(k+1)  and  P*(k)  into 

(10) .  Similarly,  IT*  (k+1) ,  r*(k+l)  and 
ftMk+1)  can  be  obtained  from  (11. b),  (17), 
and  (18. a). 

All  in  all,  the  reduced-order  optimal 
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state  estimator  is  formulated  by  the  fol*» 
lowing  equations 

•(k+l|k+I)  -  [l-K*(k+l)H(k+l)  ]F*(k)l(k|k) 

+  K*(k+1)  fy^(k+l)  -  *T*(k+l) 

z,(k+l)  -  H(k+l)G#<k)*,(lc)l 

(20. a) 

^1(k+l|k+l)  ■  ft  ( k+ 1  j  k+ 1)  +  P*(k+1)  *2( k+1) 

(20. b) 

z.(k+l)  -  C^tk+l)  y_(k+l) 

2  22  2  (20. c) 

V~(k+l|k+l)  *=  [I-K*  (k+1)  H(k+1)  jr*  (k+1) 

z  (20. d) 

K*(k+1)  =  r*(k+l)HT(k+l)  [H(k*l)r*(k+1) 


HT(k+l)  +  Vv^(k+lJ.]_1 


(20. e) 


r*(k+l)  *=  F*  (k) V~(k|  k) P*T(k)  +  M*(k)Vu(k) 

M*T(k)  =  (I:-P*(k+1)] [A  (k) 
V-tklkJ^fk)  +  S(k)Vu(k)ST(k)] 


[I:-F*(k+1) 


(20.  f) 


rvk|w!  V-a  1 

V~(k|k)  -  -  + - To1-- 

1  n^xm^  J  m2xm2 

due  to  the  fact  that  z^(k)  is  exactly 

measurable  for  every  k.  Also,  the  error 

* 

covariance  matrix  V^(k+l|k+l)  is  given  by 

V~(k+1  | k+1)  -  [(rMk+l))"1  +  HT(k+l) 

Vy^(k+1)  H(k+1))-1  (20. g) 

and  P*(k+1)  is  specified  by  Bqs.  (19)* 

4.  COMMENTS  ON  THE  REDUCED-ORDER 
OPTIMAL  ESTIMATOR 


The  formulation  for  the  optimal  state 
estimator  derived  in  last  section,  (19)- 
(20),  is  identical  to  that  of  the  "hybrid 
estimator"  given  in  [8],  except  that  in 
[8]  a  deterministic  input  is  inserted  to 
the  system  dynamics.  However,  the 
approach  here  is  more  straightforward  and 
it  is  clearer  here  that  the  choice  of 
P(k+1)  is  crucial  to  the  optimality  of  the 
estimator.  It  can  be  seen  that  the  gen¬ 
eral  coordinate  transformation  discussed 
in  [8]  is  split  into  two  coordinate  trans¬ 
formations:  one  which  depends  on  the  eye- 
tern  output  matrix  H  only  and  one  which 
depends  on  the  matrices  and  P^2*  It 

can  be  seen  that  the  similarity  transfor¬ 


mation  Q(-)  of  (3)  is  equivalent  to  J“  (•) 
defined  in  (14)  of  (8],  and  thus  the  state 
variable  z  defined  in  (3)  can  be  regarded 
as  equivalent  to defined  in  (14)  of  [8]. 
Furthermore,  the  "variable  s  defined  in  (9) 
can  be  similarly  related  to  the  variable 
in  (20)  of [8). 

The  relations  in  (19)  which  govern 
the  choice  of  P*(k+1)  are  vital  to  the 
understanding  of  the  optimal  state  esti¬ 
mator,  and  thus  deserves  some  detailed 
discussion.  First,  notice  that 

h(A1(k))  c  h(A2(k)) 

where  h(A1(k))  and  h(A2(k))  denote  the  null 
spaces  of  A1(k)  and  A2 (k) ,  respectively. 

The  following  observation  is  thus  made: 
Observation  The  set  I* (k+1)  given  in  (19. a) 
is  a  non-empty  set.  Moreover,  if  A^(k)  is 

singular .  any  member  P(k+1)  c  I* (k+1)  yields 
the  same  estimator  performance. 

Now,  according  to  the  value  of  A^(k), 

the  following  special  cases  are  of  interest: 


Case  1:  A^(k) 


In  this  case. 


and  thus 


1*  (k+1) 


"l*m2 


This  case  is  possible  if  (8)  does  not  con¬ 
tain  any  information  pertaining  to  the 
estimation  of  z^k)  ;  for  example,  if 


An  ex- 


*21W  “  °m2xni  and  B2(k)  *  °m2xp-  ex" 

treme  example  for  this  case  ie  that  m2  = 

0,  i.e.  all  measurements  are  noise  corrupt¬ 
ed.  In  this  condition,  the  estimator  pre¬ 
sented  in  Section  3  is  identical  to  the 
standard  full^order  Kalman  filter  whose 
performance  is  independent  of  the  choice 
of  P(k+1)  . 

Ca8e_2:  Aj(k)  a  0  ,  i.e.  A^k)  i*  a 

singular  non- zero  matrix.  In  this  case, 
only  some  components  of  z2(k)  contain  in¬ 
formation  about  (u(k),  z^(k)).  Thus  the 

similarity  transformation  discussed  in 
Section  2  can  be  redefined  so  as  to  iso¬ 
late  only  those  elements  of  z2(k)  which 

constitute  a  constraint  on  (u(k) , z1(k) ) . 

Hence  the  Lagrange  multiplier  P(k+1)  that 
should  be  considered  is  an  element  in 

XT  —  *  .  &  1  iamif  \  1  u 


where  r  <  m. 


Alternatively, 


any  member  in  £*(k+l)  can  be  used  in  the 
filter  realization. 

Case  3 1  A^(k)  is  positive- definite.  This 
condition  can  be  fulfilled  when  Vu(k)  is 

positive-de finite  for  any  k  ■  0,1,2,...,. 
In  this  case,  I*  (k+1)  contains  one  and 
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only  one  element  P*(k+1),  which  la  given 
by  ( 19 .  d)  . 

When  P*(*)  i®  uniquely  specified 
(Case  3) ,  one  can  compare  the  error 
covariance  matrix  given  by  (20. g)  with 
that  obtained  for  arbitrary  P(*)  and  ob¬ 
serve  the  same  expression  for  V~(k+1 |k+l) . 

The  difference  is  that  r*(k+l)  of  (20.f) 
has  the  following  property 

Tr  [T*  (k+1)  ]  <  Tr  (F(k+1)  ] 

where  T(*)  is  obtained  from  non- optimal 
P(-). 

Finally,  the  implementation  of  this 


estimator 

should  be  initiated  as  follows: 

$(0|0)  -  E[x(0) ]  -  x0 

i.e . 

^(OlO)  =  E[zx(0)] 

therefore 

v~(o|o)  ■= 

'v  (0)  10  I 

O2-  -  !  0i-2-  =  Q(0,Vx  °T(0) 
ntjXm^  |  m2xm2  J  0 

and 

P* ( 0)  -  0n 

n^x  m2 

5.  CONCLUSION 

A  reduced-order  optimal  state  esti¬ 
mator  for  a  linear,  discrete- time  system 
associated  with  a  singular  noise  covari¬ 
ance  matrix  has  been  derived  in  this  paper. 
The  main  idea  in  this  derivation  is  to 
cast  this  singular  state  estimation  problem 
as  a  constrained  optimization  problem. 

The  estimator  derived  here  is  fundamental¬ 
ly  the  same  as  that  derived  by  Fairman  [8] . 
The  major  differences  are:  the  app  oach 
here  is  more  straightforward,  the  optimal¬ 
ity  of  the  estimator  is  more  explicitly 
exposed  and,  furthermore,  the  possibility 
of  nonuniqueness  of  P* ( • )  i®  discussed 
here. 

It  is  worth  mentioning  that  the  esti¬ 
mator  given  here  requires  lower  order 
matrix  inversion  than  the  standard  full- 
order  Kalman  filter  does  in  the  singular 
case;  thus  the  computational  efficiency 
is  improved.  This  estimation  procedure 
can  be  applied  similarly  to  smoothing  and 
predicting  problems  or  systems  with  colored 
noise . 
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